The Joint Picture Experts Group (JPEG) Standard suggests storing entropy decoded Discrete Cosine Transform coefficients for all the components in memory for an image bitstream encoded in progressive JPEG Mode. As soon as a subset of frequency coefficients of all the components as partitioned by an image or JPEG encoder becomes available, the same is stored and decoded and an image that is a coarse approximation of an original image is displayed. Discrete Cosine Transform (DCT) coefficients which have been decoded are stored as they are required for decoding an improved approximation of an image after a remaining portion of the bitstream has been received. As more frequency coefficients of all the components as partitioned by image/JPEG encoder become available, they are stored and decoded along with previously stored frequency coefficients and an image, which is an improvement over the previous coarse approximation of an original image, is displayed.
JPEG image decoding as described above requires an intermediate memory of the order of image_width*image_height*no_of_components*frequency_coefficient_num_bytes for storing DCT coefficients of all image components. no_of_components is the total number of different components which when combined represent a multi-component JPEG image; for example, a YUV JPEG image consists of three components Y, U and V. frequency_coefficient_num_bytes is the number of bytes required to represent a frequency coefficient. The JPEG library developed by the independent JPEG group requires so much intermediate memory for decoding a progressively encoded JPEG bitstream.
If the original image dimensions are 2K*2K, then approx 2K*2K*3*2 bytes of memory are needed for YUV 4:4:4 color format, which is approximately 24 MB. Successive improvements in an image require repeated Inverse Discrete Cosine Transform (IDCT) computation, hence increasing computation resources by a factor proportional to the number of successive improvements.
Researchers have proposed various solutions toward efficient implementation of progressive JPEG decoder.
U.S. Pat. No. 7,313,281B2, entitled “Method and Related Apparatus for JPEG Decoding” to Chi-Cheng Ju, et. al., teaches decoding each of the scans into partial decoded pixel and summing each newly generated partial decoded pixel.
U.S. Patent Application No. 2003/0091240A1, entitled “Method and Apparatus for Progressive JPEG Image Decoding” to Chi-Cheng Ju et. al., and U.S. Patent Application No. 2007/0098275A1, entitled “Method and Apparatus for Progressive JPEG Image Decoding” to Kun-Bin Lee, teach decoding a progressive JPEG image by dividing each of the scans into multiple regions and then decoding the regions individually. Finally, the decoded coefficients of the current decoding region of all scans are outputted in order to construct a portion of the image data.
U.S. Patent Application No. 2005/0008234A1, entitled “Process and Functional Unit for the Optimization of Displaying Progressively Coded Image Data” to Uwe-Erik Martin, teaches a method for optimizing the downloading of progressively coded image data. The wait times between the time points of directly consecutive decoding steps are calculated using statistical image quality parameters of received partial image data in such a manner that the decoding steps which do not lead to perceptible improvement in the quality of a reconstructed image are suppressed.
U.S. Patent Application No. 2006/0067582 A1, entitled “Progressive JPEG Decoding System” to Mi Michael Bi, et. al., teaches a method in which DCT coefficients in a particular decoded scan are classified into two categories, namely, most significant DCT coefficients and least significant DCT coefficients. The least significant DCT coefficients are not stored directly in the memory. They are binarized and represented by either “0” or “1” indicating if they are zero or non-zero coefficients. The binarized bitmap for the least significant DCT coefficients and the actual values of most significant DCT coefficients are stored in the memory and, thus, the overall memory requirements are significantly reduced.
U.S. Patent Application No. 2008/0130746A1, entitled “Decoding a Progressive JPEG Bitstream as a Sequentially Predicted Hybrid Video Bitstream” to Soroushian, et. Al, teaches generating an intermediate bitstream by parsing a JPEG bitstream carrying a picture. The intermediate bitstream generally includes one or more encoded frames each representing a portion of the picture. A second circuit may be configured to (i) generate one or more intermediate images by decoding the encoded frames, and (ii) recreate the picture using the intermediate images.
U.S. Patent Application No. 2008/0310741A1, entitled “Method for Progressive JPEG Image Decoding” to Yu-Chi Chen, et. al., describes a method of using a non-zero history table and a sign table of each Variable Length Decoding (VLD) result, which are recorded and used as a reference for decoding the next scan layer. The decoded coefficients are no longer directly stored in a memory to save the memory space.
U.S. Patent Application No. 2009/0067732A1, entitled “Sequential Decoding of Progressive coded JPEGS” to Sukesh V. Kaithakapuzha, teaches progressive scan encoded JPEGS are decoded sequentially on a Minimum Coded Unit (MCU) basis. Address Pointers are used to index into each scan, and coded data from each scan is outputted to form an entropy decoded MCU.
Each of these attempts to solve the problem addressed by this disclosure have the similar shortcoming of increased memory usage, and increased decode latency. Secondly the prior art references perform IDCT, data copy and color format conversion, such as YCbCr to RGB, for entire MCU/data unit for every reconstruction of an approximation of an image.